Autonomous Theory Building Systems

We are interested in very general systems which are programmed once and which from then on learn autonomously all sorts of things simply by observing a sequence of input data. In this preliminary note we give examples of techniques which apparently permit to deal with two basic aspects related to such systems: basic drive and complexity of learning steps. A system which learns autonomously must have a criterion by which to decide what is worth learning. This criterion provides the basic drive for the system. We study very general criteria of this nature. They are related to the concept of building theories about the input data. A theory about any set of data D is for us simply an algorithm t which reproduces the data. The theory t is nontrivial if the length of t is less than the length of the data D (both measured in the same unit, say bits). We will focus on two aspects of the complexity of nontrivial theories t: the amount of time it takes to find t and the length of t. In sections 2 and 4 we will formally define theory building systems as well as some concepts of teaching and learning related to these systems. Sections 3, 5, 6 and 7 contain some techniques for constructing such systems. We feel that with proper teaching remarkably much can be learned by such systems with large but not astronomical computational effort.